Low awareness of the transitivity assumption in complex networks of interventions: a systematic survey from 721 network meta-analyses.

BACKGROUND: The transitivity assumption is the cornerstone of network meta-analysis (NMA). Violating transitivity compromises the credibility of the indirect estimates and, by extent, the estimated treatment effects of the comparisons in the network. The present study offers comprehensive empirical evidence on the completeness of reporting and evaluating transitivity in systematic reviews with multiple interventions. METHODS: We screened the datasets of two previous empirical studies, resulting in 361 systematic reviews with NMA published between January 2011 and April 2015. We updated our evidence base with an additional 360 systematic reviews with NMA published between 2016 and 2021, employing a pragmatic approach. We devised assessment criteria for reporting and evaluating transitivity using relevant methodological literature and compared their reporting frequency before and after the PRISMA-NMA statement. RESULTS: Systematic reviews published after PRISMA-NMA were more likely to provide a protocol (odds ratio (OR): 3.94, 95% CI: 2.79-5.64), pre-plan the transitivity evaluation (OR: 3.01, 95% CI: 1.54-6.23), and report the evaluation and results (OR: 2.10, 95% CI: 1.55-2.86) than those before PRISMA-NMA. However, systematic reviews after PRISMA-NMA were less likely to define transitivity (OR: 0.57, 95% CI: 0.42-0.79) and discuss the implications of transitivity (OR: 0.48, 95% CI: 0.27-0.85) than those published before PRISMA-NMA. Most systematic reviews evaluated transitivity statistically than conceptually (40% versus 12% before PRISMA-NMA, and 54% versus 11% after PRISMA-NMA), with consistency evaluation being the most preferred (34% before versus 47% after PRISMA-NMA). One in five reviews inferred the plausibility of the transitivity (22% before versus 18% after PRISMA-NMA), followed by 11% of reviews that found it difficult to judge transitivity due to insufficient data. In justifying their conclusions, reviews considered mostly the comparability of the trials (24% before versus 30% after PRISMA-NMA), followed by the consistency evaluation (23% before versus 16% after PRISMA-NMA). CONCLUSIONS: Overall, there has been a slight improvement in reporting and evaluating transitivity since releasing PRISMA-NMA, particularly in items related to the systematic review report. Nevertheless, there has been limited attention to pre-planning the transitivity evaluation and low awareness of the conceptual evaluation methods that align with the nature of the assumption.

Two-stage or not two-stage? That is the question for IPD meta-analysis projects.

Individual participant data meta-analysis (IPDMA) projects obtain, check, harmonise and synthesise raw data from multiple studies. When undertaking the meta-analysis, researchers must decide between a two-stage or a one-stage approach. In a two-stage approach, the IPD are first analysed separately within each study to obtain aggregate data (e.g., treatment effect estimates and standard errors); then, in the second stage, these aggregate data are combined in a standard meta-analysis model (e.g., common-effect or random-effects). In a one-stage approach, the IPD from all studies are analysed in a single step using an appropriate model that accounts for clustering of participants within studies and, potentially, between-study heterogeneity (e.g., a general or generalised linear mixed model). The best approach to take is debated in the literature, and so here we provide clearer guidance for a broad audience. Both approaches are important tools for IPDMA researchers and neither are a panacea. If most studies in the IPDMA are small (few participants or events), a one-stage approach is recommended due to using a more exact likelihood. However, in other situations, researchers can choose either approach, carefully following best practice. Some previous claims recommending to always use a one-stage approach are misleading, and the two-stage approach will often suffice for most researchers. When differences do arise between the two approaches, often it is caused by researchers using different modelling assumptions or estimation methods, rather than using one or two stages per se.

MA-cont:pre/post effect size: An interactive tool for the meta-analysis of continuous outcomes using R Shiny.

Meta-analysis is a widely used methodology to combine evidence from different sources examining a common research phenomenon, to obtain a quantitative summary of the studied phenomenon. In the medical field, multiple studies investigate the effectiveness of new treatments and meta-analysis is largely performed to generate the summary (average) treatment effect. In the meta-analysis of aggregate continuous outcomes measured in a pretest-posttest design using differences in means as the effect measure, a plethora of methods exist: analysis of final (follow-up) scores, analysis of change scores and analysis of covariance. Specialised and general-purpose statistical software is used to apply the various methods, yet, often the choice among them depends on data availability and statistical affinity. We present a new web-based tool, MA-cont:pre/post effect size, to conduct meta-analysis of continuous data assessed pre- and post-treatment using the aforementioned approaches on aggregate data and a more flexible approach of generating and analysing pseudo individual participant data. The interactive web environment, available by R Shiny, is used to create this free-to-use statistical tool, requiring no programming skills by the users. A basic statistical understanding of the methods running in the background is a prerequisite and we encourage the users to seek advice from technical experts when necessary.

How robust are findings of pairwise and network meta-analysis in the presence of missing participant outcome data?

BACKGROUND: To investigate the prevalence of robust conclusions in systematic reviews addressing missing (participant) outcome data via a novel framework of sensitivity analyses and examine the agreement with the current sensitivity analysis standards. METHODS: We performed an empirical study on systematic reviews with two or more interventions. Pairwise meta-analyses (PMA) and network meta-analyses (NMA) were identified from empirical studies on the reporting and handling of missing outcome data in systematic reviews. PMAs with at least three studies and NMAs with at least three interventions on one primary outcome were considered eligible. We applied Bayesian methods to obtain the summary effect estimates whilst modelling missing outcome data under the missing-at-random assumption and different assumptions about the missingness mechanism in the compared interventions. The odds ratio in the logarithmic scale was considered for the binary outcomes and the standardised mean difference for the continuous outcomes. We calculated the proportion of primary analyses with robust and frail conclusions, quantified by our proposed metric, the robustness index (RI), and current sensitivity analysis standards. Cohen's kappa statistic was used to measure the agreement between the conclusions derived by the RI and the current sensitivity analysis standards. RESULTS: One hundred eight PMAs and 34 NMAs were considered. When studies with a substantial number of missing outcome data dominated the analyses, the number of frail conclusions increased. The RI indicated that 59% of the analyses failed to demonstrate robustness compared to 39% when the current sensitivity analysis standards were employed. Comparing the RI with the current sensitivity analysis standards revealed that two in five analyses yielded contradictory conclusions concerning the robustness of the primary analysis results. CONCLUSIONS: Compared with the current sensitivity analysis standards, the RI offers an explicit definition of similar results and does not unduly rely on statistical significance. Hence, it may safeguard against possible spurious conclusions regarding the robustness of the primary analysis results.

Quantifying the robustness of primary analysis results: A case study on missing outcome data in pairwise and network meta-analysis.

Conducting sensitivity analyses is an integral part of the systematic review process to explore the robustness of results derived from the primary analysis. When the primary analysis results can be sensitive to assumptions concerning a model's parameters (e.g., missingness mechanism to be missing at random), sensitivity analyses become necessary. However, what can be concluded from sensitivity analyses is not always clear. For instance, in a pairwise meta-analysis (PMA) and network meta-analysis (NMA), conducting sensitivity analyses usually boils down to examining how 'similar' the estimated treatment effects are from different re-analyses to the primary analysis or placing undue emphasis on the statistical significance. To establish objective decision rules regarding the robustness of the primary analysis results, we propose an intuitive index, which uses the whole distribution of the estimated treatment effects under the primary and alternative re-analyses. This novel index is compared to an objective threshold to infer the presence or lack of robustness. In the case of missing outcome data, we additionally propose a graph that contrasts the primary analysis results to those of alternative scenarios about the missingness mechanism in the compared arms. When robustness is questioned according to the proposed index, the suggested graph can demystify the scenarios responsible for producing inconsistent results to the primary analysis. The proposed decision framework is immediately applicable to a broad set of sensitivity analyses in PMA and NMA. We illustrate our framework in the context of missing outcome data in both PMA and NMA using published systematic reviews.

Continuous(ly) missing outcome data in network meta-analysis: A one-stage pattern-mixture model approach.

Appropriate handling of aggregate missing outcome data is necessary to minimise bias in the conclusions of systematic reviews. The two-stage pattern-mixture model has been already proposed to address aggregate missing continuous outcome data. While this approach is more proper compared with the exclusion of missing continuous outcome data and simple imputation methods, it does not offer flexible modelling of missing continuous outcome data to investigate their implications on the conclusions thoroughly. Therefore, we propose a one-stage pattern-mixture model approach under the Bayesian framework to address missing continuous outcome data in a network of interventions and gain knowledge about the missingness process in different trials and interventions. We extend the hierarchical network meta-analysis model for one aggregate continuous outcome to incorporate a missingness parameter that measures the departure from the missing at random assumption. We consider various effect size estimates for continuous data, and two informative missingness parameters, the informative missingness difference of means and the informative missingness ratio of means. We incorporate our prior belief about the missingness parameters while allowing for several possibilities of prior structures to account for the fact that the missingness process may differ in the network. The method is exemplified in two networks from published reviews comprising a different amount of missing continuous outcome data.

Pattern-mixture model in network meta-analysis of binary missing outcome data: one-stage or two-stage approach?

BACKGROUND: Trials with binary outcomes can be synthesised using within-trial exact likelihood or approximate normal likelihood in one-stage or two-stage approaches, respectively. The performance of the one-stage and the two-stage approaches has been documented extensively in the literature. However, little is known about how these approaches behave in the presence of missing outcome data (MOD), which are ubiquitous in clinical trials. In this work, we compare the one-stage versus two-stage approach via a pattern-mixture model in the network meta-analysis using Bayesian methods to handle MOD appropriately. METHODS: We used 29 published networks to empirically compare the two approaches concerning the relative treatment effects of several competing interventions and the between-trial variance (τ2), while considering the extent and level of balance of MOD in the included trials. We additionally conducted a simulation study to compare the competing approaches regarding the bias and width of the 95% credible interval of the (summary) log odds ratios (OR) and τ2 in the presence of moderate and large MOD. RESULTS: The empirical study did not reveal any systematic bias between the compared approaches regarding the log OR, but showed systematically larger uncertainty around the log OR under the one-stage approach for networks with at least one small trial or low event risk and moderate MOD. For these networks, the simulation study revealed that the bias in log OR for comparisons with the reference intervention in the network was relatively higher in the two-stage approach. Contrariwise, the bias in log OR for the remaining comparisons was relatively higher in the one-stage approach. Overall, bias increased for large MOD. For these networks, the empirical results revealed slightly higher τ2 estimates under the one-stage approach irrespective of the extent of MOD. The one-stage approach also led to less precise log OR and τ2 when compared with the two-stage approach for large MOD. CONCLUSIONS: Due to considerable bias in the log ORs overall, especially for large MOD, none of the competing approaches was superior. Until a more competent model is developed, the researchers may prefer the one-stage approach to handle MOD, while acknowledging its limitations.

Meta-analysis of continuous outcomes: Using pseudo IPD created from aggregate data to adjust for baseline imbalance and assess treatment-by-baseline modification.

Meta-analysis of individual participant data (IPD) is considered the "gold-standard" for synthesizing clinical study evidence. However, gaining access to IPD can be a laborious task (if possible at all) and in practice only summary (aggregate) data are commonly available. In this work we focus on meta-analytic approaches of comparative studies where aggregate data are available for continuous outcomes measured at baseline (pre-treatment) and follow-up (post-treatment). We propose a method for constructing pseudo individual baselines and outcomes based on the aggregate data. These pseudo IPD can be subsequently analysed using standard analysis of covariance (ANCOVA) methods. Pseudo IPD for continuous outcomes reported at two timepoints can be generated using the sufficient statistics of an ANCOVA model, i.e., the mean and standard deviation at baseline and follow-up per group, together with the correlation of the baseline and follow-up measurements. Applying the ANCOVA approach, which crucially adjusts for baseline imbalances and accounts for the correlation between baseline and change scores, to the pseudo IPD, results in identical estimates to the ones obtained by an ANCOVA on the true IPD. In addition, an interaction term between baseline and treatment effect can be added. There are several modeling options available under this approach, which makes it very flexible. Methods are exemplified using reported data of a previously published IPD meta-analysis of 10 trials investigating the effect of antihypertensive treatments on systolic blood pressure, leading to identical results compared with the true IPD analysis and of a meta-analysis of fewer trials, where baseline imbalance occurred.

Mixed milk feeding: a systematic review and meta-analysis of its prevalence and drivers.

CONTEXT: Extensive literature is available on exclusive breastfeeding and formula-feeding practices and health effects. In contrast, limited and unstructured literature exists on mixed milk feeding (MMF), here defined as the combination of breastfeeding and formula feeding during the same period in term infants > 72 hours old (inclusion criterion). OBJECTIVE: A systematic review and meta-analysis were performed, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, on the global prevalence of MMF (primary outcome) and related drivers and practices (secondary outcomes). DATA SOURCES: The search of MMF in generally healthy populations was conducted across 6 databases, restricted to publications from January 2000 to August 2018 in English, Spanish, French, and Mandarin. DATA EXTRACTION: Two reviewers independently performed screenings and data extraction according to a priori inclusion and exclusion criteria. DATA ANALYSIS: Of the 2931 abstracts identified, 151 full-text publications were included for data extraction and 96 of those were included for data synthesis (the majority of those were cross-sectional and cohort studies). The authors summarized data across 5 different categories (feeding intention prenatally, and 4 age intervals between > 72 hours and > 6-23 months) and 5 regional subgroups. The overall prevalence of MMF across different age intervals and regions varied between 23% and 32%; the highest rate was found for the age group 4-6 months (32%; 95% confidence interval, 27%-38%); regional comparisons indicated highest MMF rates in Asia (34%), North and South America (33%), and Middle East and Africa together (36%), using a random effects meta-analysis model for proportions. Some drivers and practices for MMF were identified. CONCLUSION: MMF is a widespread feeding reality. A shared and aligned definition of MMF will help shed light on this feeding practice and evaluate its influence on the duration of total breastfeeding, as well as on infants' nutrition status, growth, development, and health status in the short and long terms. PROSPERO registration number CRD42018105337.

One-stage random effects meta-analysis using linear mixed models for aggregate continuous outcome data.

The vast majority of meta-analyses uses summary/aggregate data retrieved from published studies in contrast to meta-analysis of individual participant data (IPD). When the outcome is continuous and IPD are available, linear mixed modelling methods can be employed in a one-stage approach. This allows for flexible modelling of within-study variability and between-study effects and accounts for the uncertainty in the estimates of between-study and within-study residual variances. However, IPD are seldom available. For the normal outcome case, we present a method to generate pseudo IPD from aggregate data using group mean, standard deviation, and sample sizes within each study, ie, the sufficient statistics. Analyzing the pseudo IPD with likelihood-based methods yields identical results as the analysis of the unknown true IPD. The advantage of this method is that we can employ the mixed modelling framework, implemented in many statistical software packages, and explore modelling options suitable for IPD, such as fixed study-specific intercepts and fixed treatment effect model, fixed study-specific intercepts and random treatment effects, and both random study and treatment effects and different options to model the within-study residual variance. This allows choosing the most realistic (or potentially complex) residual variance structures across studies, instead of using an overly simple structure. We demonstrate these methods in two empirical datasets in Alzheimer disease, where an extensive model, assuming all within-study variances to be free, fitted considerably better. In simulations, the pseudo IPD approach showed adequate coverage probability, because it accounted for small sample effects.

Systematic literature review and network meta-analysis in highly active relapsing-remitting multiple sclerosis and rapidly evolving severe multiple sclerosis.

OBJECTIVE: Multiple sclerosis (MS) is a chronic, neurodegenerative autoimmune disorder affecting the central nervous system. Relapsing-remitting MS (RRMS) is the most common clinical form of MS and affects ∼85% of cases at onset. Highly active (HA) and rapidly evolving severe (RES) RRMS are 2 forms of RRMS amenable to disease-modifying therapies (DMT). This study explored the efficacy of fingolimod relative to other DMTs for the treatment of HA and RES RRMS. METHODS: A systematic literature review (SLR) was conducted to identify published randomised controlled trials in HA and RES RRMS. Identified evidence was vetted, and a Bayesian network meta-analysis (NMA) was performed to evaluate the relative efficacy of fingolimod versus dimethyl fumarate (DMF) in HA RRMS and versus natalizumab in RES RRMS. RESULTS: For HA RRMS, the SLR identified 2 studies with relevant patient subgroup data: 1 comparing fingolimod with placebo and the other comparing DMF with placebo. 3 studies were found for RES RRMS: 1 comparing fingolimod with placebo and 2 studies comparing natalizumab with placebo. NMA results in the HA population showed a favourable numerical trend of fingolimod versus DMF assessed for annualised relapse rate (ARR) and 3-month confirmed disability progression. For the RES population, the results identified an increase of ARR and 3-month confirmed disability progression for fingolimod versus natalizumab (not statistically significant). Sparse study data and the consequently high uncertainty around the estimates restricted our ability to demonstrate statistical significance in the studied subgroups. CONCLUSIONS: Data limitations are apparent when conducting an informative indirect comparison for the HA and RES RRMS subgroups as the subgroups analyses were retrospective analyses of studies powered to indicate differences across entire study populations. Comparisons across treatments in HA or RES RRMS will be associated with high levels of uncertainty until new data are collected for these subgroups.

Comparative efficacy and tolerability of pharmacological and somatic interventions in adult patients with treatment-resistant depression: a systematic review and network meta-analysis.

OBJECTIVE: Major depressive disorder (MDD) affects about 10-15% of the general population in a lifetime. A considerable number of patients fail to achieve full symptom remission despite adequate treatment and are considered treatment resistant (TRD). The current study compared the relative efficacy and tolerability of pharmacological and somatic TRD interventions by means of a Bayesian network meta-analysis. RESEARCH DESIGN AND METHODS: An electronic literature search of MEDLINE, MEDLINE In-Process, EMBASE, PsycInfo, EconLit and Cochrane Library databases for trials published between September 2003 and September 2014 was conducted. Key outcomes extracted were disease severity change from baseline, response and remission rates at various timepoints and discontinuation due to adverse events. RESULTS: Of the 3876 abstracts identified, 31 publications/randomised controlled trials (RCTs) were included in the analysis; 19 RCTs investigating 13 pharmacological interventions and 12 RCTs investigating electroconvulsive therapy (ECT) and repetitive transcranial magnetic stimulation (rTMS). The evidence synthesis investigating efficacy outcomes of TRD treatments demonstrated superior efficacy for ketamine compared to pharmacological and somatic interventions at 2 weeks after treatment initiation. At 4, 6 and 8 weeks, quetiapine augmentation (800 mg/day) and risperidone augmentation were found to be the first and second best treatments, respectively. Networks were small for response rate and remission rate outcomes at most timepoints. The most tolerable treatment was lamotrigine augmentation showing a comparable profile to placebo/sham. CONCLUSIONS: This analysis revealed scarcity of long-term data on sustained remission that would allow a comparative long-term efficacy assessment. Key limitations of the analysis can be considered the search timeframe and the use of mapping formula for the depression scores.

High correlation of VAS pain scores after 2 and 6 weeks of treatment with VAS pain scores at 12 weeks in randomised controlled trials in rheumatoid arthritis and osteoarthritis: meta-analysis and implications.

BACKGROUND: Researchers in clinical trials in rheumatoid arthritis (RA) and osteoarthritis (OA) often measure pain levels with a visual analogue scale (VAS). Of interest to clinical practice and future clinical trial design are associations of change from baseline (CFB) between time points with predictive ability of earlier response for long-term treatment benefit. We assessed the association and predictive ability of CFB in VAS pain between 2, 6 and 12 weeks in randomised controlled trials (RCTs) of non-steroidal anti-inflammatory drugs (NSAIDs). METHODS: Aggregated VAS pain data at baseline and CFB at 2, 6 and 12 weeks were collected from a systematic literature review of 176 RCTs in OA and RA. The predictive ability of earlier assessments for longer-term pain reduction was estimated using correlation and regression analyses. Analysis was performed using the R software package for statistical programming, version 3.1.1. RESULTS: Appropriate data were available from 50 RCTs (22,854 patients). Correlations between time points were high (weighted correlation coefficients between 2 and 6 weeks, 0.84; between 2 and 12 weeks, 0.79; and between 6 and 12 weeks, 0.96). CFB at 6 weeks was highly predictive and close to CFB at 12 weeks (regression coefficient 0.9, 95 % confidence interval 0.9-1.0). CFB at 2 weeks was significantly associated with CFB at 12 (0.8, 0.7-0.8) and 6 weeks (0.9, 0.8-1.0). CONCLUSIONS: The results showed that early analgesic response measured by VAS for pain beyond 2 weeks of treatment with a particular NSAID is likely to be predictive of response at 12 weeks. Failure to achieve desired pain relief in OA and RA after 2 weeks should trigger reassessment of dosage and/or analgesic.